Entropy Rate Estimation for Markov Chains with Large State Space

Estimating the entropy based on data is one of the prototypical problems in distribution property testing and estimation. For estimating the Shannon entropy of a distribution on S elements with independent samples, [Pan04] showed that the sample complexity is sublinear in S, and [VV11a] showed that consistent estimation of Shannon entropy is possible if and only if the sample size n far exceeds S log S . In this paper we consider the problem of estimating the entropy rate of a stationary reversible Markov chain with S states from a sample path of n observations. We show that (a) As long as the Markov chain mixes not too slowly, i.e., the relaxation time is at most O( S ln S ), consistent estimation is achievable when n ≫ S log S . (b) As long as the Markov chain has some slight dependency, i.e., the relaxation time is at least 1+Ω( ln 2 S